Detection device and detection method

ABSTRACT

A detection device, comprising an image sensor provided with a plurality of pixels having a plurality of light receiving sections corresponding to a micro lens, capable of adding output signals of the plurality of light receiving sections in accordance with phase difference detection direction and outputting an added signal, and a processor for performing focus detection or depth detection using phase difference detection, based on the added signal of the image sensor, whereby the image sensor makes height of a potential barrier between the light receiving sections different in accordance with the plurality of phase difference detection directions, and the processor sets charge storage determination level differently in accordance with the phase difference detection direction, and controls charge storage operation of the image sensor based on the added signal and the charge storage determination level.

CROSS-REFERENCE TO RELATED APPLICATIONS

Benefit is claimed, under 35 U.S.C. § 119, to the filing date of priorJapanese Patent Application No. 2020-149838 filed on Sep. 7, 2020. Thisapplication is expressly incorporated herein by reference. The scope ofthe present invention is not limited to any requirements of the specificembodiments described in the application.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to a detection device that has an imagesensor provided with a plurality of pixels corresponding to microlenses, obtains phase difference of an output signal using a pluralityof pixel outputs, and can perform focus detection or depth detectionbased on this phase difference.

2. Description of the Related Art

A focus detection device is known in which a plurality of micro lensesand a plurality of pixels corresponding to each micro lens are arrangedon an image plane of an image sensor, a plurality of light fluxes thatpass through regions in which exit pupils of an imaging optical systemhave been divided are respectively subjected to photoelectric conversionby each pixel, phase difference of these output signals that have beensubjected to photoelectric conversion is detected, and defocus amount ofimaging light flux is detected based on this phase difference. It isalso known to adding output signals of each pixel that has been dividedand use the resulting signal as an image signal of a single pixel.

For example, a photoelectric conversion device shown in Japanese patentlaid-open No, 2015-162658 (hereafter referred to as “patent publication1”) has a configuration with divided photodiodes (PD) being arranged incorrespondence with a single micro lens of an image sensor, and phasedifference detection in a lateral (RL) direction is performed usingpixel outputs resulting from adding electrical charge of upper and lower(TB) PDs among the four divided PDs. Also, phase difference detection ina vertical (TB) direction is performed using pixel output resulting fromadding electrical charge of lateral (RL) PDs.

If light is irradiated to a pixel of the image sensor from an angle,electrical charge generated as a result of photoelectric conversion oneach of a pair of pixels will be different, and the electrical charge ofsome pixels will be saturated. In this case, linearity of an imagesignal that has a pair of pixel signals (electrical charge) combined byaddition is lost. With patent publication 1, if electrical charge amountthat has been photoelectrically converted is saturated in any one of the4-divided PDs, phase difference detection will not be possible foreither RL or TB. As a result, with patent publication 1 a thresholdvoltage corresponding to electrical charge amount (saturation)equivalent to a potential barrier of one PD is required to be set to ¼of a normal threshold voltage.

As shown in patent publication 1, if a potential barrier between PDs ismade lower, a combined image signal is not saturated, even if somepixels (PDs) are saturated. However, with patent publication 1 since athreshold voltage must be set to ¼ of a normal threshold voltage,dynamic range of a signal for focus detection (AF) becomes insufficient,and detection accuracy is lowered.

SUMMARY OF THE INVENTION

The present invention has been conceived in view of this type ofsituation, and provides a detection device and detection method that canperform high precision and high speed focus detection or depthdetection.

A detection device of a first aspect of the present invention comprisesan image sensor provided with a plurality of pixels having a pluralityof light receiving sections corresponding to a micro lens, with theplurality of light receiving sections being provided in correspondencewith a specified plurality of phase difference detection directions,capable of adding output signals of the plurality of light receivingsections in accordance with the phase difference detection direction andoutputting an added signal, and a controller for performing focusdetection or depth detection using phase difference detection, based onthe added signal of the image sensor, whereby the image sensor makesheight of a potential barrier between the light receiving sectionsdifferent in accordance with the plurality of phase difference detectiondirections, and the controller executes a charge storage operation ofthe image sensor by instructing a specified phase difference detectiondirection to the image sensor, sets a charge storage determination levelthat differs in accordance with the phase difference detectiondirection, and controls a charge storage operation of the image sensorbased on the added signal and the charge storage determination level.

A detection method of a second aspect of the present invention is adetection method for a detection device that has an image sensorprovided with a plurality of pixels having a plurality of lightreceiving sections corresponding to a micro lens, with the plurality oflight receiving sections being provided in correspondence with aspecified plurality of phase difference detection directions, capable ofadding output signals of the plurality of light receiving sections inaccordance with the phase difference detection direction and outputtingan added signal, the detection method comprising, making height of apotential barrier between the light receiving sections different inaccordance with the plurality of phase difference detection directions,and executing a charge storage operation of the image sensor byinstructing a specified phase difference detection direction to theimage sensor, setting a charge storage determination level that differsin accordance with the phase difference detection direction, controllinga charge storage operation of the image sensor based on the added signaland the charge storage determination level, and performing focusdetection or depth detection using phase difference detection based onthe added signal of the image sensor.

A non-transitory computer-readable medium of a third aspect of thepresent invention, storing a processor executable code, which whenexecuted by at least one processor, performs a detecting method, theprocessor being provided in a detection device, the detection devicehaving an image sensor provided with a plurality of pixels having aplurality of light receiving sections corresponding to a micro lens,with the plurality of light receiving sections being provided incorrespondence with a specified plurality of phase difference detectiondirections, capable of adding output signals of the plurality of lightreceiving sections in accordance with the phase difference detectiondirection and outputting an added signal the detecting methodcomprising, making height of a potential barrier between the lightreceiving sections different in accordance with the plurality of phasedifference detection directions, executing a charge storage operation ofthe image sensor by instructing a specified phase difference detectiondirection to the image sensor, setting a charge storage determinationlevel that differs in accordance with the phase difference detectiondirection, controlling a charge storage operation of the image sensorbased on the added signal and the charge storage determination level,and performing focus detection or depth detection using phase differencedetection based on the added signal of the image sensor.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram mainly showing the electrical structure of animaging device of one embodiment of the present invention.

FIG. 2 is a block diagram mainly showing the electrical structure of animage sensor of an imaging device of one embodiment of the presentinvention.

FIG. 3A is a block diagram showing the structure of a 4PD pixel typepixel, in the imaging device relating to one embodiment of the presentinvention. FIG. 3B is a cross section along an optical axis direction ofa micro lens, for a 4PD pixel type pixel section, in the imaging deviceof one embodiment of the present invention.

FIG. 4A and FIG. 4B are drawings for describing potential barrier, of a4PD pixel type pixel section, in the imaging device of one embodiment ofthe present invention, with FIG. 4A being a case of RL pixel priority,and FIG. 4B being a case of both pixel uniformity.

FIG. 5 is a drawing showing sequences at the time of AF and at the timeof LV, in the imaging device of one embodiment of the present invention.

FIG. 6 is a drawing for describing pixel addition at the time of RLreadout, in the imaging device of one embodiment of the presentinvention.

FIG. 7 is a drawing for describing pixel addition at the time of TBreadout, in the imaging device of one embodiment of the presentinvention.

FIG. 8 is a drawing showing sequences at the time of rapid shooting, inthe imaging device of one embodiment of the present invention.

FIG. 9A to FIG. 9C are flowcharts showing operation of the imagingdevice of one embodiment of the present invention.

FIG. 10 is a block diagram showing a circuit for setting amplificationfactor of an image sensor, in an imaging device of a first modifiedexample of one embodiment of the present invention.

FIG. 11A and FIG. 11B are a graph and tables showing a relationshipbetween gain of the image sensor and a saturation threshold value, inthe imaging device of a first modified example of one embodiment of thepresent invention.

FIG. 12 is a graph and table showing a correction amount for each pixelat the time of lens fitting, in an imaging device of a second modifiedexample of one embodiment of the present invention.

FIG. 13 is a drawing showing a state where an L pixel is saturated, andelectrical charge is overflowing to an R pixel side, in an imagingdevice of a third modified example of one embodiment of the presentinvention.

FIG. 14 is a circuit diagram showing structure of a 4PD pixel, in oneembodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

An imaging device of one embodiment of the present invention will bedescribed in the following. This imaging device has an imaging section,with a subject image being converted to image data by this imagingsection, and the subject image being subjected to live view display on adisplay section arranged on the rear surface of the camera body based onthis converted image data. A photographer determines composition andphoto opportunity by looking at the live view display. At the time of arelease operation image data is stored in a storage medium. Also, imagedata that has been stored in the storage medium can be subjected toplayback display on the display section if playback mode is selected.

Also, an image sensor 208 of an imaging device of this embodiment isprovided with a plurality of pixels having a plurality of lightreceiving sections corresponding to micro lenses, and the plurality oflight receiving sections are provided in correspondence with a specifiedplurality of phase difference detection directions (refer to FIG. 3A,FIG. 3B, FIG. 6 and FIG. 7). Also, the image sensor 208 makes height ofa potential barrier between light receiving sections different dependingon phase difference detection direction (refer to FIG. 4A).

With this embodiment, specifically, 4-divided photodiodes are arrangedin correspondence with a single micro lens of the image sensor (refer toFIG. 3A), and phase difference detection is performed with prioritygiven to a horizontal direction (left right direction) over the verticaldirection (up down direction) when detecting phase difference. Regardinga potential barrier between R pixels that are arranged on the right sideand have been vertically added (hereafter, R pixels) and L pixels thatare arranged on the left side and have been vertically added (hereafter,L pixels), and a potential barrier between T pixels that are arranged onthe top and have been horizontally added (hereafter, T pixels) and Bpixels that are arranged on the bottom and have been vertically added(hereafter, B pixels), the potential barriers are suitable for RL pixelpriority (refer to FIG. 4A). Specifically, the potential barrier betweenT pixels that are arranged on the top and B pixels that are arranged onthe bottom is set lower than the potential barrier between R pixels andL pixels. (for example, the potential barrier is set to 1/2, refer toFIG. 4A). In a case of 2PD (R/L pixels or T/B pixels) readout in orderto perform horizontal (vertical) phase difference detection withvertical (left right) addition, threshold voltages (charge storagedetermination level and saturation determination level that will bedescribed later (refer, for example, to S9 to S15 in FIG. 9A, and S27 toS33 in FIG. 9B)) corresponding to height of a potential barrier of apair of pixels to be read out are set, and a charge storage operation isperformed.

In the case of 2PD readout for phase difference detection, a thresholdvoltage for saturation determination of pixel data at the time ofreadout with R/L pixels is set to a threshold voltage corresponding toelectrical charge amount for 2PDs corresponding to the potential barrierof the R/L pixels (for example, double the threshold voltage for 1PD(1/2 the threshold voltage for 4PDs)). As a result, it is possible tosimply achieve a larger dynamic range of a signal for R/L pixels withhigh readout speed and high readout efficiency, and it becomes possibleto increase AF precision and make AF high speed. It should be noted,regarding readout of R/L pixel signals corresponding to horizontal phasedifference detection, that since increase and decrease in the number oflines that are read out has little effect on phase difference detectionprecision, it is easy to reduce the number of lines that are read out bypixel addition or pixel thinning. On the other hand, regarding readoutof T/B pixel signals corresponding to vertical phase differencedetection, since increase or decrease in the number of lines that areread out has a significant effect on phase difference detectionprecision, a reduction in the number of lines that are read out willlower precision. For this reason, with readout of R/L pixel signals itis simple to make readouts be high-speed while maintaining precision,and readout efficiency is higher.

FIG. 1 is a block diagram showing one example of the structure of animaging device (specifically, a digital camera, for example) 1 thatincludes a focus detection device of one embodiment of the presentinvention. It should be noted that in FIG. 1 solid line arrows show flowof data, and dashed line arrows show flow of control signals.

An imaging device 1 comprises an interchangeable lens 100 and a camerabody 200. The interchangeable lens 100 is configured so that it ispossible to attach to the camera body 200. When the interchangeable lens100 is attached to the camera body 200, the interchangeable lens 100 andthe camera body 200 are connected so that communication is possiblebetween them. It should be noted that the imaging device 1 is notnecessarily a lens interchangeable imaging device. For example, theimaging device 1 may be a lens integrated imaging device. The imagingdevice may also be provided within a portable device, such as a smartphone.

The interchangeable lens 100 comprises an imaging optical system 102, adrive section 104, a lens CPU (Central Processing Unit) 106, and a lensside storage section 108. Here, each block of the interchangeable lens100 is configured using hardware, for example. However, all blocks donot necessarily have to be configured using hardware, and some sectionsmay be configured using software. Also, each block of theinterchangeable lens 100 need not be configured using a single hardwareor software component, and may be configured using a plurality ofhardware or software components. Also, in a case where theinterchangeable lens and the camera body are integrated, the lens CPU106 and the CPU 212 may be configured as a single CPU.

The imaging optical system 102 is an optical system imaging light fluxfrom a subject on to the image sensor 208 of the camera body 200. Theimaging optical system 102 comprises a focus lens 102 a and an aperture102 b. The focus lens 102 a is constructed so as to be able to adjustfocal position of the imaging optical system 102 by moving in an opticalaxis direction.

The aperture 102 b is arranged on the optical axis of the focus lens 102a. The opening diameter of the aperture 102 b is variable. The aperture102 b adjusts amount of light from a subject passing through the focuslens 102 a that is incident on the image sensor 208. The drive section104 has a drive motor and a drive circuit etc., and drives the focuslens 102 a and the aperture 102 b based on control signals output fromthe lens CPU 106. Here, the imaging optical system 102 may be configuredas a zoom lens. In this case, the drive section 104 may also performzoom drive, and focal length may also be changed by manual operation ofthe user. The drive section 104 functions as an aperture drive section(actuator, driver) for driving the aperture that is included in theimaging optical system.

The lens CPU 106 is a processor that includes a CPU and peripheralcircuits for the CPU, and operates in accordance with programs stored ina lens side storage section 108. The lens CPU 106 is configured so as tobe able to communicate with the CPU 212 of the camera body 200 via aninterface (I/F) 110. The lens CPU 106 controls the drive section 104 inaccordance with control signals from the CPU 212 of the camera body 200.Also, the lens CPU 106 transmits various information, such as aperturevalue (F value) of the aperture 102 b, and lens information etc. storedin the lens side storage section 108, to the CPU 212 via the I/F 110.The lens CPU 106 functions as a focus lens control section that controlsposition of the focus lens contained in the imaging optical system. Thisfocus lens control section communicates with a control section toexecute position control of the focus lens in synchronization with afirst imaging operation and second imaging operation of the imagesensor.

It should be noted that the lens CPU 106 is not necessarily configuredas a CPU. That is, functions that are the same as those of the lens CPU106 may also be implemented using a processor such as an ASIC(Application Specific Integrated Circuit) or FPGA (Field-ProgrammableGate Array) etc. Also, functions that are the same as those of the lensCPU 106 may also be implemented using software.

The lens side storage section 108 is an electrically rewritablenonvolatile memory, and stores lens information etc. relating to theinterchangeable lens 100, as well as the above described programs. Lensinformation includes, for example, focal length information andaberration information of the imaging optical system 102.

The camera body 200 comprises a mechanical shutter 202, a drive section204, an operation section 206, the image sensor 208, a hand shakecorrection circuit 210, the CPU 212, an image processing circuit 214, animage compression and expansion section 216, a focus detection circuit218, an exposure control circuit 220, a display section 222, a bus 224,DRAM (Dynamic Random Access Memory) 226, a body side storage section228, and a storage medium 230. Here, each block of the camera body 200is configured using hardware, for example. However, all blocks andcircuits do not necessarily have to be configured using hardware, andsome sections may be configured using software. Also, each block of thecamera body 200 need not be configured using a single hardware orsoftware component, and may be configured using a plurality of hardwareor software components.

The mechanical shutter 202 has an opening and closing structure, andadjusts a time for which light flux from the subject is incident on theimage sensor 208 (exposure time of the image sensor 208). A focal planeshutter, for example, is adopted as the mechanical shutter 202. Besidesthis focal plane shutter, a lens shutter may be provided at the lensbarrel side. The drive section 204 drives the mechanical shutter 202based on control signals from the CPU 212. The drive section 204comprises an actuator that drives the mechanical shutter 202, and drivecircuitry etc. for this actuator, and performs opening and closingoperations of the mechanical shutter 202.

The operation section 206 is an interface for inputting userinstructions to the imaging device 1, and includes various operationmembers such as various operation buttons like a power supply button,release button, movie button, mode dial, playback button, menu button,etc. and a touch panel etc. This operation section 206 detects operatingstate of the various operation members, and outputs signals representingdetection results to the CPU 212.

The image sensor 208 is arranged on the optical axis of the imagingoptical system 102, at a position that is behind the mechanical shutter202, and where light flux from a subject is formed into an image by theimaging optical system 102. The image sensor 208 images a subject andgenerates a pixel signal relating to the subject.

The image sensor 208 has a pixel section 22 (refer to FIG. 2) with aplurality of imaging pixels arranged two-dimensionally. Imaging pixelsare constructed divided into a plurality of focus detection pixels,corresponding to a microlens L (refer to FIG. 3A and FIG. 3B). The focusdetection pixels generate photoelectric conversion signals byrespectively subjecting light flux, that passes through regionsresulting from subjecting a plurality of exit pupils of an imaging lens2, which is an imaging optical system, to pupil-division, tophotoelectric conversion. While the image sensor 208 is constructed as asingle CMOS image sensor provided with a primary color Bayer array colorfilter, this structure is not limiting.

The image sensor 208 is an image sensor that is provided with aplurality of pixels having a plurality of light receiving sections(light receiving regions) corresponding to a micro lens, with theplurality of light receiving sections (light receiving regions) beingprovided in correspondence with a specified plurality of phasedifference detection directions, capable of adding output signals of theplurality of light receiving sections in accordance with the phasedifference detection direction and outputting an added signal. The abovedescribed pixels can output a pixel signal for every light receivingregion. The image sensor makes height of a potential barrier between thelight receiving sections different in accordance with the plurality ofphase difference detection directions (refer, for example, to FIG. 4A).The image sensor is capable of amplifying and outputting an added signal(refer, for example, to FIG. 2, and to the analog processing section 23of FIG. 10). The detailed structure of the image sensor 208 will bedescribed later using FIG. 2, FIG. 3A and FIG. 3B.

The hand shake correction circuit 210 moves the image sensor 208 indirections parallel to the light receiving surface of the image sensor,so as to suppress camera shake that has been generated in the camerabody 200. By moving the image sensor 208 so as to negate camera shakemovement, blurring of the subject image occurring in image data that isattributable to camera shake is suppressed. It should be noted that thecamera shake correction circuit may be provided in the interchangeablelens 100. A camera shake correction circuit in this case is configuredso as to move a camera shake correction optical system that is includedin the imaging optical system 102.

The CPU 212 is a processor that includes a CPU and peripheral circuitsfor the CPU, and performs overall control of the camera body 200 inaccordance with programs stored in a body side storage section 228. TheCPU 212 controls imaging operations (imaging drive mode, readout modeetc.) by the image sensor 208, for example. Also, the CPU 212 outputscontrol signals for driving the focus lens 102 a to the lens CPU 106, inaccordance with focus state of the focus lens 102 a that has beendetected by the focus detection circuit 218. The CPU 212 also outputsexposure setting values that have been calculated by the exposurecontrol circuit 220 to the lens CPU 106 and the image sensor 208. Here,the CPU 212 is not necessarily configured as a CPU. Specifically,functions that are the same as those of the CPU 212 may also beimplemented using an ASIC or FPGA etc. Also, functions that are the sameas those of the CPU 212 may also be implemented using software.

The image processing circuit 214 applies various image processing topixel data. For example, at the time of still picture shooting (alsoincluding rapid shooting), the image processing circuit 214 appliesimage processing for still picture storage and generates still picturedata. Similarly, at the time of movie shooting, the image processingcircuit 214 applies image processing for movie storage and generatesmovie data. Further, at the time of live view display the imageprocessing circuit 214 applies image processing for display andgenerates display image data.

The image compression and expansion section 216 has an image compressioncircuit and an image expansion circuit. At the time of image datastorage, the image compression and expansion section 216 compressesimage data that has been generated by the image processing circuit 214(still picture data or movie data). Also, at the time of image dataplayback, image data that is stored in the storage medium 230 in acompressed state is expanded.

The focus detection circuit 218 performs focus detection for the focuslens 102 a using a phase difference method that uses focus detectionpixel data output from the focus detection pixels of the image sensor208. Also, the focus detection circuit 218 is capable of detecting depthof an object using phase difference detection results.

The focus detection circuit 218 functions as a control section(controller) that performs focus detection or depth detection usingphase difference detection based on an added signal of an image sensor.It should be noted that functions of this control section (controller)are not limited to the focus detection circuit 218, and may also betaken on by other circuits and processors, such as the CPU 212 etc. Thecontrol section (controller) instructs a specified phase differencedetection direction to the image sensor to execute a charge storageoperation of the image sensor (refer, for example, to S5 in FIG. 9A andS23 in FIG. 9B), sets charge storage determination level that isdifferent depending on phase difference detection direction (refer, forexample, to S9 and S13 in FIG. 9A and S27 and S31 in FIG. 9B), andcontrols the charge storage operation of the image sensor based on anadded signal and the charge storage determination level.

The above described control section (controller) also sets the chargestorage determination level in accordance with height of a potentialbarrier corresponding to the phase difference detection direction(refer, for example, to S9 and S13 in FIG. 9A, and S27 and S31 in FIG.9B). The control section (controller) compares the added signal and thecharge storage determination level, and controls charge storageoperation of the image sensor based on the comparison result (refer, forexample, to S9 and S13 in FIG. 9A and to S27 and S31 in FIG. 9B).

The above described control section (controller) also sets amplificationfactor of the image sensor and sets charge storage determination levelin accordance with the amplification factor that has been set (refer,for example, to FIG. 11A and FIG. 11B). The control section (controller)sets a saturation determination level in accordance with heights of apotential barrier corresponding to the phase difference detectiondirection, and determines saturation by comparing the added signaloutput by the image sensor with the saturation determination level(refer, for example, to FIG. 11A and FIG. 11B). The control section(controller) performs focus detection or depth detection based on thedetermination result.

Also, in a case where added signals of a plurality of light receivingsections corresponding to phase difference detection direction areacquired by causing imaging by the image sensor, between continuousshooting of still pictures, the control section (controller) reads outonly added signals corresponding to a phase difference detectiondirection for which potential barrier has been set higher, as addedsignals of a plurality of light receiving section corresponding to phasedifference detection direction (refer to FIG. 8, for example).

The exposure control circuit 220 fulfills a function as a photometrysection, and calculates exposure setting values based on pixel data ofthe image sensor 208. This exposure control circuit 220 measures subjectbrightness from pixel data of the image sensor 208, and calculatesexposure setting values necessary to make brightness of the subject atthe time of shooting a correct value, from the subject brightness thathas been measured. Exposure setting values include opening amount of theaperture 102 b (aperture value) and exposure time of the image sensor208 (shutter speed).

The display section 222 has a display such as a liquid crystal displayor an organic EL display, and is arranged on a rear surface etc. of thecamera body 200, and functions as an electronic viewfinder. This displaysection 222 displays images in accordance with control by the CPU 212.The display section 222 is used in live view display, and in playbackdisplay of already stored images etc.

The bus 224 is connected to the image sensor 208, CPU 212, imageprocessing circuit 214, image compression and expansion section 216,focus detection circuit 218, exposure control circuit 220, displaysection 222, DRAM 226, body side storage section 228 and storage medium230, and operates as a transfer circuit for transferring various datathat has been generated by these blocks.

The DRAM 226 is an electrically rewritable volatile memory, andtemporarily stores various data such as pixel data output from the imagesensor 208, still picture data, movie data, display image data, andprocess data for the CPU 212 etc. It should be noted that it is alsopossible to use an SDRAM (synchronous dynamic random access memory) astemporary storage.

The body side storage section 228 is an electrically rewritablenon-volatile memory. The body side storage section 228 stores variousdata such as programs used by the CPU 212 and adjustment values for thecamera body 200 etc. The storage medium 230 is an electricallyrewritable non-volatile memory, and is built into the camera body 200 orconfigured to be loaded into the camera body 200. The storage medium 230stores image data for storage as an image file of a specified format. Itshould be noted that the DRAM 226, body side storage section 228, andstorage medium 230 may be respectively configured as a single memory, ormay be configured as a combination of a plurality of memories etc.

Next, the structure of the image sensor 208 will be described using FIG.2. The image sensor 208 has image pixels, and these image pixels aredivided into a plurality of focus detection pixels. Image pixelsgenerate an image pixel signal and a focus detection pixel signal basedon a photoelectric conversion signal that has been generated byphotoelectric conversion of light flux by focus detection pixels. Itshould be noted that image pixels of this embodiment use photodiodes aslight receiving sections.

In the example shown in FIG. 2, the image sensor 208 comprises avertical scanning section 21, a pixel section 22, an analog processingsection 23, an ADC processing section 24, memory section 25, horizontalscanning section 26, output section 27, input section 28, and elementcontrol section 29.

Image pixels and focus detection pixels are arranged in the pixelsection 22. The image pixel signals and focus detection pixel signalsare generated by photoelectric conversion of a subject image, and readout of these signals that have been generated is performed by at leastone section among the vertical scanning section 21 to output section 27,and the element control section 29 etc.

The vertical scanning section 21 has a vertical scanning circuit, andperforms scanning in a vertical direction by successively selectingpixel rows (lines) in a horizontal direction within the pixel section22. This vertical scanning section 21 selects a particular line, andcontrols charge accumulation time of pixels (exposure time) byperforming resetting and transfer of each pixel of the line that hasbeen selected.

The analog processing section 23 has an analog processing circuit, andis a circuit for subjecting an analog pixel signal that has been readout from the pixel section 22 to analog signal processing. This analogprocessing section 23 includes, for example, a preamp that amplifies thepixel signal, and a correlated double sampling (CDS) circuit thatsubtracts reset noise from the pixel signal, etc.

The analog digital conversion processing section (ADC processingsection) 24 has an A/D conversion circuit, and converts the analog pixelsignal that has been output from the analog processing section 23 todigital pixel data. This ADC processing section 24 adopts a structure,such as exemplified by column ADC, for example, whereby a pixel signalthat has been read out from the pixel section 22 is subjected to ADconversion by an analog to digital converter (ADC) for every column.

The memory section 25 has a memory, and is configured by an electricallyrewritable volatile memory circuit etc. that temporarily holds pixeldata that has been converted by the ADC processing section 24. Thehorizontal scanning section 26 has a horizontal scanning circuit, andreads out pixel data (image pixel data and focus detection pixel data)from the memory section 25 in the order of columns.

The output section 27 has an output circuit, and organizes pixel signalsthat have been read out from the horizontal scanning section 26 forgenerating pixel signal rows, converts to an output signal format suchas a serial signal or differential signal etc. and outputs the convertedresult. It should be noted that this output section 27 or the abovedescribed ADC processing section 24 etc. function as a sensitizationsection that performs sensitization processing (signal amplificationprocessing in accordance with ISO sensitivity that has been set).

The input section 28 has an input circuit, and receives synchronizationsignals, a reference clock, and operation setting information etc.relating to control of the image sensor 208 from the CPU 212 and anot-illustrated image sensor drive section.

The element control section 29 has an imaging control circuit, and isfor controlling each block within the image sensor 208 in conformitywith synchronization signals and a reference clock that have beenreceived via the input section 28, and is provided with a readout methodselection section 30. Also, the element control section 29 receivesoperation setting instructions, such as instructions for switchingimaging drive mode, from the CPU 212 via the input section 28, andcontrols each block within the image sensor 208.

The readout method selection section 30 has a selection circuit, andselects and sets a readout method for readout from the image sensor 208based on operation setting information (for example, camera modes suchas still picture shooting, movie shooting, live view, AF etc.) that hasbeen received via the input section 28. As readout methods some or allof a 1PD simple readout system, a 2PD additive readout system, a 4PDadditive readout system with no phase difference information for readingout all added values of focus detection pixels, etc. may be set. Also,AF readout (AF (rl)), which will be described later, generates and readsout both of a pair of focus detection pixel signals (R/L pixel signals)for a first pupil-division direction based on a photoelectric conversionsignal, using a 2PD additive read out method. Live view (LV) readout(LV+AF (tb)) generates and reads out both of a pair of focus detectionpixel signals (T/B pixel signals) for a second pupil-division directionbased on a photoelectric conversion signal, using a 2PD additive readout method. Actual exposure readout is generation of an image pixelsignal by addition of all photoelectric conversion signals that havebeen generated within a single image pixel, using a 4PD additive readoutmethod, and reading out of only the image pixel signal that has beengenerated.

Next, the structure of the focus detection pixels and image pixelsarranged in the pixel section 22 will be described using FIG. 3A andFIG. 3B. As has been described above, the pixel section 22 is a pixelarray section having image pixels and focus detection pixels arrangedtwo dimensionally (in the vertical direction (column direction) andhorizontal direction (line direction)).

FIG. 3A shows an example of a 4PD pixel structure, and is a pixelstructure having four photodiodes PD arranged in a single micro lens Le.The 4PD pixel shown in FIG. 3A has a single color filter F and fourphotodiodes, PDa, PDb, PDc, and PDd arranged for a single micro lens Le.Each pixel is configured with a micro lens Le, color filter F andphotodiodes PDa to PDd arranged sequentially in a lamination directionfrom an object side to an imaging surface, as shown in FIG. 3B. Themicro lens Le is for increasing light amount reaching the 4PD pixels, asimage pixels, by concentrating light, and effectively making an apertureratio of the image pixels large. Regarding the color filter F, in acase, for example, of a primary color Bayer array color filter, eitherof an R filter, G filter or B filter is provided in accordance with thatpixel position.

With the 4PD pixel shown in FIG. 3A and FIG. 3B, photodiodes PDa to PDdare arranged in a pupil-division direction in the imaging range of thesingle micro lens Le, and the four photodiodes PDa to PDd are divided into four, namely top, bottom, left, and right, so that it is possible todetect phase difference in the horizontal direction and verticaldirection. The four photo diodes PD are respectively arranged at upperleft, lower left, upper right, and lower right positions. Specifically,a single pixel has four photodiodes PDa, PDb, PDc and PDd, and there aretwo pupil-division directions, namely the horizontal direction and thevertical direction.

In a case where outputs of photodiodes PD are subjected to vertical 2PDaddition, namely, in a case where (PDa+PDb) and (PDc+PDd) in FIG. 3A aregenerated, this will constitute focus detection pixel signals fordetecting phase difference in the horizontal direction (vertical linedetection). As shown at the upper right of FIG. 3A, a left side 2PDaddition value L and a right side 2PD addition value R are obtained.Also, in a case where outputs of photodiodes PD are subjected tohorizontal 2PD addition, namely, in a case where (PDa+PDc) and (PDb+PDd)are generated, this will constitute focus detection pixel signals fordetecting phase difference in the vertical direction (horizontal linedetection). As shown at the lower right of FIG. 3A, an upper 2PDaddition value T and a lower 2PD addition value L are obtained. Also, inthe case where outputs of photodiodes PD are subjected to 4PD addition,namely in a case where (PDa+PDb+PDc+PDd) is generated, a pixel signal of4PD addition constitutes an image pixel signal

A 4PD pixel structure and PD addition will be described using thecircuit diagram shown in FIG. 14. Switching transistors Tr1-Tr4 arerespectively connected to the four photodiodes PDa-PDd. If controlsignals TX1-TX4 from the vertical scanning section 21 are respectivelyapplied to the switching transistors Tr1-Tr4, the transistors Tr1-Tr4are selectively turned on. If the switching transistors Tr1-Tr4 areturned on, the photodiodes PDa to PDd and a floating diffusion FD areconnected, and signal charge of a photodiode PD corresponding to atransistor Tr that has been turned on is transferred to the floatingdiffusion FD. For example, it is possible to transfer a signal chargeresulting from addition of signal charges for PDa and PDb to the FD byturning on Tr1 and Tr2.

Also, one end of a switching transistor Tr5 connected to a point ofconnection of the switching transistors Tr1-Tr4 and the floatingdiffusion FD, with the other end of the transistor Tr5 being connectedto a power supply voltage VDD. If a reset signal RES is applied to Tr5,the power supply voltage VDD and the FD are connected to reset the FD.By turning the switching transistor Tr5 on in a state where theswitching transistors Tr1 to Tr4 are on, reset of the photodiodes PDa toPDd is performed. The floating diffusion FD is connected to an outputterminal OUT via a switching transistor Tr6, and an amplifyingtransistor Tr7 that is connected to the power supply voltage VDD. If aselection signal SEL is applied to the switching transistor Tr6, avoltage value of the floating diffusion FD is amplified by transistorTr7, and output to the output OUT, and this output voltage is input tothe analog processing section 23.

Also, although described in detail using FIG. 4A and FIG. 4B, a firstboundary B2 (corresponding to the barrier Ptb in FIG. 4A) exists betweenthe pair of photodiodes PDa and PDb, and between the pair of photodiodesPDc and PDd, and a second boundary B1 (corresponding to the barrier Prlin FIG. 4A) exists between the pair of photodiodes PDa and PDc, andbetween the pair of photodiodes PDb and PDd. A potential barrier of thissecond boundary B1 is set so as to become higher compared to thepotential barrier of the first boundary B2.

Next, setting of the potential barrier of a 4PD pixel will be describedusing FIG. 4A and FIG. 4B. Height of a potential barrier between lightreceiving sections (PD) is made different depending on the plurality ofphase difference detection directions, but since an R/L pixel has abetter readout efficiency than a T/B pixel, with this embodiment the R/Lpixel is used with priority.

FIG. 4A shows height of a potential barrier (hereafter simply calledbarrier) between a pixel and a light receiving section PD in the case ofRL pixel priority. In FIG. 4A, LT and LB respectively represent an upperleft light receiving section LT (PDa), and a lower right light receivingsection LB (PDb), while RT and RB respectively represent an upper rightlight receiving section RT (PDc) and a lower right light receivingsection RB (PDd). As was described using FIG. 3A, in a case wheredetection direction of phase difference is arrow A (also called RLdirection), it is possible to generate an L pixel signal by addingelectrical charges of LT and LB, and it is possible to generate an Rpixel signal by adding electrical charges of RT and RB. In the case ofR/L pixel priority, level of a barrier Ptb (corresponding to boundaryB2) that is positioned between LT and LB, and positioned between RT andRB is set so as to become low with respect to level of a barrier Prl(corresponding to boundary B1) that is positioned between LT and RT, andpositioned between LB and RB.

In this way, since level of barrier Ptb is set so as to become low withrespect to the level of barrier Prl, incident light amount on to LTbecomes larger than for other light receiving sections, for example, andthere may be cases where an electrical charge that has been accumulatedexceeds the barrier Ptb and spills over. In this case, this electricalcharge that has spilled over flows into LB. A case where electricalcharge exceeds the barrier and spills over in this way is known assaturation. Electrical charge that has been generated in the L pixel(LT+LB) is limited to the barrier Prl, and there is no flowing out ofelectrical charge into the R pixel (RT+RB). As a result of this, in acase where phase difference is detected in phase difference direction A,saturation of electrical charge does not occur in either of the R pixel(RT+RB) or L pixel (LT+LB), even if saturation of electrical chargeoccurs in any one of the light receiving sections LT, LB, RT, RB withrespect to the barrier Ptb. Accordingly, since there is no saturationwith respective pixel values of R pixels and L pixels (pixel outputvoltage), it is possible to detect phase difference for the phasedifference direction A. Next, description will be given of the effect ofhaving priority, compared to a case where there is no priority relatingto barriers.

In a case where there is no priority relating to the barrier, as shownin FIG. 4B, specifically when level of the barrier Pa is equal in the RLdirection and the TB direction, then in the event that any one of thelight receiving sections LT, LB, RT, and RB has an accumulatedelectrical charge that is saturated and exceeds the barrier Pa, causingspillover of electrical charge, a balance of electrical charge amountcollapses for both the RL direction phase difference and the TBdirection phase difference, and phase difference detection becomesimpossible. In order to avoid this type of electrical charge saturation,it is necessary to adjust accumulation time by monitoring accumulatedcharge amount. With the image sensor 208 electrical charge that has beenaccumulated in a PD is transferred to a capacitance (floatingdiffusion), and converted to a voltage to be read out as a signalvoltage. This capacitance is set to a capacitance value that has beenoptimized as an image pixel signal, which is a 4PD addition pixelsignal. If a voltage signal corresponding to charge saturation of a 4PDaddition pixel signal is made 1, then a voltage signal corresponding tocharge saturation of the light receiving sections LT, LR, RT and RBbecomes 1/4.

Therefore, a 1PD pixel signal is set to a threshold voltage levelcorresponding to a charge amount corresponding to level of a 1PD barrierPa, namely set to a threshold voltage level of 1/4 of the thresholdvoltage level corresponding to charge saturation of a 4PD addition pixelsignal, and it is necessary to control accumulation time so that thisthreshold voltage level is not exceeded. In a case where there is notthis type of priority with levels of barriers Pa made equal, as athreshold voltage level it is necessary to make a value of 1/4 of athreshold voltage level for a 4PD addition pixel signal an electricalcharge saturation determination level. The same applies in the case of a2PD addition pixel signal, and in order to determine saturation ofelectrical charge for 1PD it is necessary to make a value of 1/4 of athreshold voltage level of a 4PD addition pixel signal a chargesaturation determination level. Accordingly, in a case where there is nopriority for barriers, a dynamic range of signal voltage of phasedifference detection pixels (R/L pixels, T/B pixels) is reduced to 1/4of the dynamic range of signal voltage of image pixels (4PD addition),and there is a problem in that precision of phase difference detectionis lowered.

On the other hand, if there is priority with respect to barriers, suchas the RL pixel priority that has been described above, in a case whereR/L pixels are read out, if electrical charge amount for 2PDs LT and LBdoes not exceed barrier Prl, then even if electrical charge amount of asingle PD LT (or LB) among the 4PD exceeds barrier Ptb respective signalvoltages of the R/L pixels will not be saturated. In a case where levelof a barrier Prl between R/L pixels is made the same as a barrier Pa forthe case where there is no priority, an electrical charge amount for2PDs (LT and LB, or RT and RB) corresponding to the level of the barrierPa for the R/L pixels becomes 1/2 the electrical charge amount for 4PDaddition corresponding to the level of the barrier Pa. Accordingly, forthe signal voltage of the R/L pixels it is possible to set an electricalcharge saturation determination level to 1/2 of the electrical chargesaturation determination level for the signal voltage for 4PD addition.As a result of this, it is possible to make the dynamic range of thesignal voltage for RL direction phase difference detection pixels (R/Lpixels) 1/2 of the dynamic range of the signal voltage of image pixels(4PD addition). It is possible to enlarge the dynamic range by twocompared to the case where there is no priority. It should be noted thatsetting of a potential barrier can be performed by adjusting impurityconcentration of a semiconductor in a manufacturing process of the imagesensor, and detailed description is omitted.

Next, a method of setting saturation determination level for pixel datawill be described based on setting of barrier level that was describedusing FIG. 4A. In the case of reading out pixel data for R/L pixels withRL pixel priority, the saturation determination level is set to avoltage level corresponding to level of a barrier Prl between the Rpixels and L pixels, as was described previously. Generally, level ofthe barrier Prl is set to 70% of the saturation level for electricalcharge of 4PD addition pixels, in order to ensure linearity of the 4 PDaddition pixel. In a case where saturation level for signal voltage in acase of reading out a pixel signal for a 4PD addition pixel is made, forexample, 4096 (corresponding to 12 bit quantization), saturation levelfor signal voltages of R/L pixels corresponds to signal voltagecorresponding to electrical charge amount for 2 respective PDs, which is4096×70%/2=1433. Accordingly, saturation determination level for signalvoltage of R/L pixels (pixel data) may be set to, for example, about1300 (refer to S9 in FIG. 9A, and S27 in FIG. 9B).

In a case of reading out pixel data for T/B pixels, the saturationdetermination level is set to a voltage level corresponding to the levelof a barrier Ptb between T pixels and B pixels. Level of the barrier Ptbis set to 70% of the level of the barrier Prl. In a case wheresaturation level for signal voltage in a case of reading out a pixelsignal for a 4PD addition pixel is made 4096, saturation level forsignal voltages of T/B pixels corresponds to 4096×70%×70%/4=501, whichis signal voltage corresponding to electrical charge amount for 2respective PDs. Accordingly, saturation determination level for signalvoltage of T/B pixels (pixel data) may be set to, for example, about 450(refer to S13 in FIG. 9A, and S31 in FIG. 9B).

On the other hand, in the case where RL pixel priority is not set, ifsaturation of electrical charge occurs for any one of the four PDs therewill be imbalance, which will have an effect on correlation calculation.In this case, under the same conditions, saturation level for signalvoltage of R/L pixels will become 4096×70%/4=716. Accordingly, bysetting RL pixel priority it is possible to ensure dynamic range ofabout double (1433/716) for signal voltage of R/L pixels (pixel data),and the effect of this is significant.

In a frame in which the phase difference detection shown in FIG. 8,which will be described later, is performed, saturation is determined bysetting a saturation determination level for R/L pixels in the case ofexposure/readout of R/L pixel signals, and setting a saturationdetermination level for T/B pixels in a case of exposure/readout of T/Bpixel signals (refer, for example, to S9 to S15 in FIG. 9A, and S27 toS33 in FIG. 9B).

FIG. 5 is a timing chart for describing exposure and readout processingfor AF and LV (live view) (refer, for example, to S5 in FIG. 9A and S23in FIG. 9B). “VD” represents timing of a synchronization signal(vertical synchronization signal) input to the element control section29. VD(1) is a vertical synchronization signal indicating timing of anAF operation that includes imaging and readout for AF, and VD(2) is avertical synchronization signal indicating timing of an LV operationthat includes imaging and readout for LV. VD(1) and VD(2) are input onceat a time to the element control section 29 between single frames, andprocessing for exposure and readout for AF and LV is performed for everyone frame. Length of one frame is determined in accordance with updateinterval of a screen of an LV display.

In FIG. 5, an AF operation shown by AF (rl) is an operation forgenerating and reading out focus detection pixel signals relating to afirst pupil-division direction. The first pupil-division direction isthe horizontal direction of the pixel section 22 (RL direction), forexample. Before an AF operation the element control section 29 switchessettings of the pixel section 22 so that R/L pixel signals, which are aleft opening pixel signal l (Rl, Grl, Gbl, Bl) and a right opening pixelsignal (Rr, Grr, Gbr, Br), are output from the pixel section 22. Theelement control section 29 performs setting for pixel addition, foradding (averaging) a plurality of R/L pixel signals for the same colorand same opening that are output from the pixel section 22, in order toshorten pixel signal readout time. This pixel addition (averaging) isperformed by the analog processing section 23.

Next, pixel addition (averaging) of R/L pixels for horizontal phasedifference detection, that have been read out from the pixel section 22in an AF (rl) operation, will be described using FIG. 6. The upperdrawing in FIG. 6 shows an image of arrangement corresponding to eachpixel. Rl positioned at (m1, n1) means a left side opening red pixel(R), and Rr positioned at (m2, n1) means a right side opening red pixel(R). Gbl positioned at (m1, n2) means a left side opening green pixel(Gb), and Gbr positioned at (m2, n2) means a right side opening greenpixel (Gb). “B” means a blue pixel.

Pixel addition for R/L pixels is set such that in the analog processingsection 23 addition of only vertical direction pixels, which are asecond pupil-division direction, is performed, without performingaddition for the horizontal direction, which is the first pupil-divisiondirection. For the horizontal direction setting is performed such thatpixel signals for the same openings (associated left openings orassociated right openings) are subjected to 1/1 pixel addition (noaddition), and in the vertical direction setting is performed so thatpixel signals for the same openings (associated left openings orassociated right openings) are subjected to 5/9 pixel addition. 5/9pixel addition means adding of 5 pixels R1 corresponding to n1 to n9,and the addition of 5 pixels Gbl corresponding to n2 to n10, among 9pixels in the vertical direction (with the example of m1 in FIG. 6, 9respective pixels for Rl corresponding to n1 to n17, and for Gblcorresponding to n2 to n18). A number of additions of pixel signals maybe set appropriately in accordance with frame rate, for example. Anumber of rows of pixel signals is decreased by compressing a number ofrows of pixel signals as a result of this setting, and pixel signalreadout time is shortened. On the other hand, detection precision ofphase difference in the horizontal direction is ensured since number ofcolumns of pixel signals is not compressed.

After the CPU 212 has output control signals to the image sensor 208(element control section 29) and set pixel addition mode, as shown inFIG. 6, control signals are output to the image sensor 208 (elementcontrol section 29) so as to perform imaging with an exposure timerequired to generate focus detection pixel signals. This exposure timeis set based on subject brightness and saturation state of pixel dataetc.

The element control section 29 receives input of control signals fromthe CPU 212 and commences imaging (electrical charge accumulation) forevery line of the pixel section 22, and controls the vertical scanningsection 21 to sequentially output RL pixel signals from the pixelsection 22 for lines for which imaging has been completed.

In order to increase speed and reduce power consumption, detection ofhorizontal direction phase difference is performed using, as R/L pixelsignals, a pair made up of Gr left side opening pixel signal Grl andright side opening pixel signal Grr, and a pair made up of Gb left sideopening pixel signal Gbl and right side opening pixel signal Gbr. It isnot always necessary to read out a pair made up of R left side openingpixel signal Rl and right side opening pixel signal Rr, and a pair madeup of B left side opening pixel signal Bl and right side opening pixelsignal Br. At the time of readout, only pairs of Grl and Grr, and pairsof Gbl and Gbr, may be read out, as shown in the lower drawing of FIG.6.

The lower drawing in FIG. 6 in a conceptual diagram of data arrangementof a memory section 25 for pixel data that has been subjected to A/Dconversion after pixel addition. Addition values Gr_L for lines of leftside opening Grl pixels n1, n3, n5, n7, n9 of column m3 in the upperdrawing of FIG. 6 (in FIG. 6, in order to reduce space, pixels aredescribed only as “Gr”) are arranged in column m1 in the lower drawingof FIG. 6, and addition values Gr_R of right side opening Grr pixels n1,n3, n5, n7, n9 of column m4 in the upper drawing of FIG. 6 (in FIG. 6,in order to reduce space, pixels are described only as “Gr”) arearranged in column m2. Also, similarly, addition values Gb_L for linesof left side opening Gbl pixels n2, n4, n6, n8, n10 of column m1 in theupper drawing of FIG. 6 (in FIG. 6, in order to reduce space, pixels aredescribed only as “Gb”) are arranged in column m1 in the lower drawingof FIG. 6, and addition values Gb_R of right side opening Gbr pixels n2,n4, n6, n8, n10 of column m2 in the upper drawing of FIG. 6 (in FIG. 6,in order to reduce space, pixels are described only as “Gb”) arearranged in column m2. As shown in the lower drawing of FIG. 6, otheradded pixel data is similarly arranged.

In this way, for each line, addition values Gr_L, Gr_R, Gb_L and Gb_Rfor 5 pixels of Grl and Grr pixels, or Gbl and Gbr pixels, are stored inthe memory section 25 as R/L pixel signals, in the format shown in thetable in the lower drawing of FIG. 6. The image sensor 208 outputs theseaddition values Gr_L and Gr_R, and Gb_L and Gb_R as R/L pixel signals,and correlation calculation etc., which will be described later, isperformed in the focus detection circuit 218 using this pixel data. Thepair of Rl and Rr, and the pair of Bl and Br, may both be read outtogether.

Here, processing associated with inside the image sensor 208 will bedescribed. R/L pixel signals that have read out from the pixel section22 are subjected to analog processing such as CDS and gain processing,pixel addition processing etc. in the analog processing section 23. R/Lpixel signals after pixel addition that have been subjected to analogprocessing are converted to R/L pixel data after pixel addition, whichare digital signals in the ADC processing section 24, and stored in thememory section 25. The horizontal scanning section 26 receives controlsignals from the element control section 29 and transfers R/L pixel dataafter pixel addition (the lower drawing in FIG. 6) that are stored inthe memory section 25 to the output section 27 in order of columns.

The output section 27 generates pixel data lines by arranging pixel datathat has been transferred by the horizontal scanning section 26 intolines, and converts the pixel data lines that have been generated to aspecified output signal format, such as a serial signal or adifferential signal, and outputs the result. These pixel data lines arestored in the DRAM 226 in order of arrangement, such as shown in thelower drawing of FIG. 6, as R/L pixel data after pixel addition (focusdetection pixel data rl). It should be noted that pixel addition hasbeen described as being executed in the analog processing section 23,but pixel addition may also be executed in the ADC processing section 24or the memory section 25 without performing pixel addiction in theanalog processing section 23.

Next, description will be given of data processing for AF, associatedwith an AF (rl) operation. Pixel data lines that have been stored in theDRAM 226 as a result of a readout operation by the AF (rl) operation(R/L pixel data after pixel addition) are used in correlationcalculation for the purpose of focus deviation amount calculation. Thefocus detection circuit 218 performs correlation calculation using pairsof Grl and Grr, and pairs of Gbl and Gbr, which are R/L pixel data afterpixel addition stored in the DRAM 226 (focus detection pixel data rlshown in the lower drawing of FIG. 6).

Next, description will be given of the live view (LV) operation shown inFIG. 5 described above. The LV operation shown as LV+AF (tb) is mainlyan operation for generating and reading out a display pixel signal, butgeneration and readout of a focus detection pixel signals relating tothe second pupil-division direction in the LV operation is alsoperformed. Before an LV operation, in accordance with instructions fromthe CPU 212 the element control section 29 switches settings of thepixel section 22 so that upper opening pixel signals t (Rt, Grt, Gbt,Bt) and lower opening signals b (Rb, Grb, Gbb, Bb), that are T/B pixelsignals from the pixel section 22, are output. Also, the element controlsection 29 performs setting for pixel addition of T/B pixel signals thatare output from the pixel section 22 in order to shorten readout time ofpixel signals.

Next, description will be given of pixel addition (averaging) in the LVoperation, using FIG. 7. With an LV operation, setting is made such thatpixel addition is performed in both a horizontal direction, being afirst pupil-division direction, and a vertical direction, being a secondpupil-division direction. FIG. 7 is a conceptual diagram of a similararrangement to FIG. 6, but with left and right arrangement of T pixelsignals and B pixel signals, and not vertical arrangement.

In the upper drawing of FIG. 7, setting is performed so that pixelsignals (associated T pixel signals or associated B pixel signals) ofthree of the same openings aligned in a horizontal direction (associatedupper openings or associated lower openings) are added, and pixelsignals (associated T pixel signals or associated B pixel signals) oftwo of the same openings aligned in a vertical direction (associatedupper openings or associated lower openings) are added.

2/3 pixel addition in the vertical direction is addition of 2 pixelsfrom among 3 pixels of the same color in the vertical direction, andwith column m1 in FIG. 7 is addition of two pixels n1 and n5 among threepixels for Rt corresponding to n1 to n5, and addition of two pixels n4and n8 among three pixels for Gbt corresponding to n4 to n8. 3/3 pixeladdition in the horizontal direction is addition of three pixels of thesame color that are adjacent in the horizontal direction, and withcolumn n1 in FIG. 7 is addition of three adjacent Rt pixels of m1, m5,and m9 (adjacent Rb pixels of m2, m6 and m10). Also, three adjacent Grtpixels m7, m11 and m15 (adjacent Grb pixels m8, m12 and m16) are added.

In detecting phase difference in the vertical direction, a number ofadded pixels in the vertical direction is made two pixels in order toensure detection resolution in the vertical direction. Although it ispossible to further improve detection resolution by doing away withpixel addition in the vertical direction, a number of lines read out isincreased and readout speed is lowered (readout efficiency is low), andso 2/3 pixel addition is set, number of lines read out is decreased to1/3, and readout time is shortened. In FIG. 7, it is shown that additionis not performed for the second line and the third and fourth columns,but this arises simply because a number of lines and a number of columnsof the pixel section 22 are not multiples of three.

After the CPU 212 has transmitted control signals to the image sensor208 (element control section 29) and set pixel addition mode, as shownin FIG. 7, control signals are output to the image sensor 208 (elementcontrol section 29) so as to perform imaging with an exposure timerequired to generate focus detection pixel signals. This exposure timeis set based on subject brightness and saturation state of pixel dataetc.

Here, processing associated with an LV operation within the image sensor208 will be described. The element control section 29 receives input ofcontrol signals from the CPU 212 and commences imaging (electricalcharge accumulation) for every line of the pixel section 22, andcontrols the vertical scanning section 21 to sequentially output T/Bpixel signals from the pixel section 22 for lines for which imaging hasbeen completed. T/B pixel signals that have been read out from the pixelsection 22 are subjected to analog processing in the analog processingsection 23, converted to T/B pixel data after pixel addition, which aredigital signals, in the ADC processing section 24, and stored in thememory section 25 in an arrangement as shown in the lower drawing inFIG. 7, as T/B pixel signals after pixel addition.

With the LV operation, in order to generate display image data, pixeldata for R, Gr, Gb and B is required, and upper opening pixel data (Tpixel data) Rt, Grt, Gbt and Bt, and lower opening pixel data (B pixeldata) Rb, Grb, Gbb and Bb are stored in the memory section 25. Thehorizontal scanning section 26 receives control signals from the elementcontrol section 29 and transfers T/B pixel data after pixel additionthat are stored in the memory section 25 to the output section 27 inorder of columns. The output section 27 generates pixel data lines byarranging T/B pixel data after pixel addition that has been transferredby the horizontal scanning section 26 into lines, and converts the pixeldata lines that have been generated to a specified output signal formatsuch as a serial signal or a differential signal, and outputs theresult.

These pixel data lines that have been output from the image sensor 208are stored in the DRAM 226 as T/B pixel data after pixel addition (focusdetection pixel data), in a format such as shown in the lower drawing ofFIG. 7. It should be noted that the pixel addition in FIG. 7 has beendescribed as being executed by the analog processing section 23, but thepixel addition may also be executed by the ADC processing section 24 andthe memory section 25 instead of being executed by the analog processingsection 23.

Next, description will be given of data processing for AF (tb)associated with an LV operation. Pixel data lines that have been storedin the DRAM 226 as a result of a readout operation by the LV operation(T/B pixel data after pixel addition) is used in correlation calculationfor the purpose of focus deviation amount calculation. The focusdetection circuit 218 performs correlation calculation using pairs ofGrt and Grb, and pairs of Gbt and Gbb, which are T/B pixel data afterpixel addition stored in the DRAM 226 (focus detection pixel data tbshown in the lower drawing of FIG. 7). Correlation calculation usingpairs of Rt and Rb, and pairs of Bt and Bb may also be executed asrequired. Description will also be given of data processing for LVassociated with the LV operation. For pixel data lines that have beenstored in the DRAM 226 as a result of readout operation by the LVoperation (T/B pixel data after pixel addition), T pixel data and Bpixel data for the same color and same position are added by the imageprocessing circuit 214, and image data for LV display having R, Gr, Gb,and B pixel data is generated. The CPU 212 performs LV display with thedisplay section 222 using the image data for LV display.

Next, the actual exposure (still picture frame) and AF exposure (phasedifference frame) in high speed rapid shooting mode will be describedusing FIG. 8. FIG. 8 shows operation executed during still picture rapidshooting using an electronic shutter, in rapid shooting mode andcontinuous AF (CAF) mode. For example, the user operates the operationsection 206 to set the above-described modes. Also, in a state where a2nd release switch has been turned on by the user, there is a rapidshooting operation with the mechanical shutter 202 in a fully openstate. The imaging device of this embodiment is capable of continuousshooting of still pictures, based on an added signal resulting fromhaving added output signals of all of a plurality of light receivingsections of the image sensor.

In FIG. 8, VD represents timing of a vertical synchronization signal foran imaging operation, and T1 to T13 in the horizontal axis directionrepresent times. At time T1 initial (still picture first frame) actualexposure (corresponding to still picture and live view SI & LV in FIG.8) is commenced, at time T2 exposure for the first line is completed,and after that actual exposure for the second frame commences at timeT4, actual exposure for the third frame commences at time T7, and actualexposure for the fourth frame commences at time T10.

In respective actual exposures, still picture image data of only imagepixel signals is acquired by reading out 4PD addition values of pixelsignals of photodiodes PDa to PDd of a 4PD pixel (refer to FIG. 3A),from the image sensor 208. This image data is processed and stored asstill picture image data, together with processing as image data forlive view display and execution of live view display.

Also, between one actual exposure and another actual exposure, pixeldata of focus detection pixels for phase difference detection are readout. At time T3 exposure for initial (first frame) phase differencedetection (corresponding to phase difference detection PDD in FIG. 8) iscommenced, exposure for the first line is completed at time T4, andafter that exposure for phase difference detection of the second framecommences at time T6, and exposure for phase difference detection of thethird frame commences at time T9. In respective exposures for phasedifference detection, added values PDa+PDb, PDc+PDd (for horizontalphase difference), which are R/L pixel signals of a pixel signal of 4PDdivided photodiodes (refer to FIG. 3A), and added values PDa+PDc,PDb+PDd (for vertical phase difference), which are T/B pixel signals ofphotodiodes, are readout as image data for phase difference detection.

With this practical example, a potential barrier for RL pixel priorityis set in the image sensor 208, and readout processing is performed withpriority given to R/L pixel signals that have a larger dynamic range.That is, with exposure for phase difference detection focus detectioncalculation is performed by reading out only R/L pixel signals. Also,exposure control (accumulation time control) is performed so that datasaturation is avoided, based on the R/L pixel signals, as will bedescribed later. The AF (rl) operation shown in FIG. 5 describedpreviously may also be executed as an exposure and readout operation forR/L pixel signals. By using only R/L pixel signals that have a widerdynamic range, it is possible to shorten the time for a phase differencedetection exposure and readout operation, and to make a focus detectionoperation highly accurate, and a higher speed rapid shooting operationbecomes possible.

Also, with each phase difference detection, exposure and readout ofpixel signals for both horizontal phase difference and vertical phasedifference may be continuously executed. Also, both the previouslydescribed AF (rl) operation, as exposure and readout of pixel signalsfor horizontal phase difference, and the previously described LV+AF (tb)operation, as exposure and readout of pixel signals for vertical phasedifference, may be continuously executed. Also, exposure and readout maybe performed by alternately switching between pixel signals forhorizontal phase difference and vertical phase difference, such that thephase difference detection exposure between time T3 and time T4 performsexposure and readout of pixel signals for horizontal phase difference,while the phase difference detection exposure between time T6 and timeT7 performs exposure and readout of pixel signals for vertical phasedifference. Also, exposure and readout for horizontal phase differenceand vertical phase difference may be set for every pixel, and exposureand readout may be executed simultaneously for both pixel signals in oneframe. In this way image data for phase difference detection isacquired.

The camera body 200 performs communication by outputting a lenssynchronization signal, for synchronizing to imaging synchronizationsignal VD, to the interchangeable lens 100. This lens synchronizationsignal may have period aligned with the imaging synchronization signalVD, and may have a period that is double the imaging synchronizationsignal VD, as shown in FIG. 8. With this example, VD is output at timesT1, T2, T4, T5, T7, T8, T10, T11 and T13. On the other hand, the lenssynchronization signal is output at times t1, t4, t7, t10 and t13, andhas double the period of VD. It should be noted that as well as havingthe same period as VD, and double the period of VD, the lenssynchronization signal may also have another period. Phase of the lenssynchronization signal may coincide completely with the phase of theimaging synchronization signal VD, as shown in FIG. 8, or may bedisplaced.

The camera body 200 transmits focus lens drive amount (direction), thathas been calculated based on results of phase difference detection, tothe lens 100 in synchronism with the lens synchronization signal. Forexample, the camera body 200 (CPU 212) calculates focus lens driveamount (direction) based on results of phase difference detection for VDof T3 to T4, and after transmission of lens synchronization signal t4 tothe lens 100 transmits the focus lens drive amount (direction). The lensCPU 106 drives the focus lens based on the focus lens drive amount(direction) that has been received.

Next, description will be given for operation of the imaging device thatincludes the focus detection device, using the flowcharts shown in FIG.9A to FIG. 9C. This flow shows operation when AF mode of the imagingdevice 1 has been set to continuous AF, and low speed rapid shootingmode, which is a rapid shooting mode different to that in FIG. 8, and inwhich still picture shooting (actual exposure) is performed using amechanical shutter, has been set. Continuous AF mode is an AF mode thatis suitable for a moving subject, and is an AF mode in which focusing isperformed continuously so as to track a subject. For example, the useroperates the operation section 206 to set the above-described modes.This flowchart is realized by the CPU 212 within the camera body 200controlling each section within the camera body 200 and theinterchangeable lens 100 in accordance with programs that have beenstored in the body side storage section 228.

If it is detected that the user has performed an ON operation of thepower supply of the imaging device 1, the flow for camera power supplyON shown in FIG. 9A is commenced. If the power supply ON operation isdetected, it is first determined whether or not a 1st release switch ison (S1). Here, the CPU 212 determines whether or not a 1st releaseswitch of a release button within the operation section 206 is in an onstate. If the user focuses on a subject and decides on exposure, therelease button is pressed down halfway. If the user performs ahalf-press operation of the release button, the 1st release switchenters an on state in response to this operation.

If the result of determination in step S1 is that the 1st release switchis not on, live view (LV) exposure and readout is performed (S3). Here,the CPU 212 outputs a control signal to the drive section 204 so as toput the mechanical shutter 202 in a fully-open state, as well asoutputting a control signal to the lens CPU 106 so as to move theaperture 102 b by a given amount (for example, open aperture wider).After that, the CPU 212 outputs control signals for the image sensor 208every predetermined time (time determined by display frame rate), andperforms imaging for LV display using the image sensor 208. Every timeimaging for LV display is completed, the element control section 29reads out pixel signals from the pixel section 22. It should be notedthat at the time of pixel signal readout, the element control section 29adds pixel signals of the same opening (same color) output from thepixel section 22, and outputs the resulting signal. Pixel data that hasbeen output from the image sensor 208 is stored in the DRAM 226 as datafor display.

If the pixel data for display has been stored in step S3, the CPU 212next performs live view (LV) display, and generates display image datain the image processing circuit 214. The image processing circuit 214performs necessary processing on the pixel data that has been read outfrom the image sensor 208, to generate display image data for display.Display image data is obtained by additive averaging of pixel data ofphotodiodes PDa to PDd that belong to the same pixel section 22 (microlens). This additive averaging processing is not limited to beingexecuted in the image processing circuit 214, and may also be executedwithin the image sensor 208. If the CPU 212 has performed LV display todisplay an LV image on the display section 222 based on display imagedata that has been generated by the image processing circuit 214, S1 isreturned to.

If the result of determination in step S1 is that the 1st release switchis on, exposure and readout for AF and LV are performed (S5). The CPU212 performs the imaging and readout for autofocus (AF) and LV displaythat was described in FIG. 5. Imaging and readout for AF performs AFoperation AF (rl), and imaging and readout for LV display performs LVoperation (LV+AF (tb)). Focus detection pixel data rl, tb that wascalculated in step S5 is stored in DRAM 226, and is used at the time ofcorrelation calculation using a phase difference method. Also, the imageprocessing circuit 214 generates pixel data for display by adding dataconstituting pairs of focus detection pixels data tb that has beenstored in the DRAM 226, and this pixel data for display that has beengenerated is stored in DRAM 226. The CPU 212 performs live view displaybased on the pixel data for display that has been stored in the DRAM226.

Next, correlation calculation and reliability determination areperformed (S7). Here, the CPU 212 executes focus detection calculationusing the focus detection circuit 218. The focus detection circuit 218performs correlation calculation using focus detection pixel data rl, tbthat constitute a pair, that is stored in the DRAM 226. Correlationcalculation is the respective generation of two-image interval valuescorresponding to phase difference, for AF areas, based on focusdetection pixel data within a plurality of AF areas (focus detectionregions). Once correlation calculation is completed, the focus detectioncircuit 218 performs reliability determination for focus detection.Reliability determination is determination based on reliabilityevaluation value corresponding to contrast of a subject image acquiredfrom pixel data, and a plurality of correlation values calculated asresults of correlation calculation.

If reliability determination has been performed, next saturationdetection for R/L pixel data is performed (S9). It is detected whetheror not R/L pixel data that was read out as focus detection pixel data inAF exposure and readout (AF (rl)) is saturated. Comparison of pixel dataand a specified threshold value TH that has been set for R/L pixels (asone example of saturation determination level, 1300) is performed, andif the pixel data is larger than the specified threshold value(corresponding to signal amount being long) it is determined that thepixel data is saturated.

If the result of determination in step S9 is that the R/L pixel data issaturated, an exposure correction amount Tv_RL+ is set to 1 (S11). Theexposure correction amount Tv_RL+ that has been set is used incorrection of exposure time with the AF&LV exposure and readout of stepS5 the next time (FIG. 5: AF (rl)). Exposure time corresponding to thenext R/L pixels is set to the exposure time that has been increased by 1Ev (exposure time of 1/2), and exposure control is performed so thatthere is no saturation. In one example, 1 Ev may be a change amount of0.5 Ev or 2 Ev. Also, positions of R/L pixels for which it has beendetected that pixel data is saturated are stored in DRAM 226. Positioninformation on these data saturated pixel is used in order to determinein which focus detection region of a plurality of focus detectionregions data saturated pixels are contained (area selection of stepS17). In this way, exposure time is set so that R/L pixel data is notsaturated, and the saturation determination level (threshold value TH)is also called charge storage determination level as it is used as atarget accumulation level.

If exposure correction amount has been set in step S11, or if the resultof determination in step S9 was that saturation of R/L pixel data wasnot detected, saturation detection for T/B pixel data is performed(S13). In an LV operation (LV+AF (tb)), it is detected whether or notT/B pixel data that has been read out as focus detection pixel data issaturated. Comparison of pixel data and a specified threshold value THthat has been set for T/B pixels (as one example of saturationdetermination level, 450) is performed, and if the pixel data is largerthan the specified threshold value (corresponding to signal amount beinglarge) it is determined that the pixel data is saturated.

If the result of determination in step S13 is that the T/B pixel data issaturated, an exposure correction amount Tv_TB+ is set to 1 (S15). Theexposure correction amount Tv_TB+ that has been set is used incorrection of exposure time with the AF&LV exposure and readout of stepS5 the next time (FIG. 5: LV+AF (tb)). That is, exposure timecorresponding to the next T/B pixels is set to an exposure time that hasbeen increased by 1 Ev (exposure time of 1/2), and exposure control isperformed so that there is no saturation. In one example, 1 Ev may be achange amount of 0.5 Ev or 2 Ev. Also, positions of T/B pixels for whichit has been detected that pixel data is saturated are stored in DRAM226. Position information on these data saturated pixels is used inorder to determine in which focus detection region of a plurality offocus detection regions data saturated pixels are contained (areaselection of step S17). In this way, exposure time is set so that T/Bpixel data is not saturated, and the saturation determination level(threshold value TH) is also called charge storage determination levelwhen used as an accumulation level that is made a target.

If exposure correction amount has been set in step S15, or if the resultof determination in step S13 was that saturation of T/B pixel data wasnot detected, then next, area selection is performed (S17). Here, thefocus detection circuit 218 calculates focus deviation amount (defocusamount) of the focus lens 102 a using a two-image interval value of afocus detection region that was determined to have high reliability as aresult of reliability determination, and selects a focus detectionregion (and corresponding defocus amount) based on this focus deviationamount. For example, a focus detection region exhibiting a defocusamount corresponding to the closest subject distance (closest range) isselected. Area selection processing is not limited to the closestsubject, and it is also possible to select an area in which a person'sface exists, and it is also possible have an area that has been selectedmanually by the user. It should be noted that area selection may also beperformed by the CPU 212. In the area selection, focus detection regionsthat contain a specified number or more of either R/L pixels or T/Bpixels for which data saturation has been detected are made invalid andremoved from targets of area selection. This is because focus detectioncalculation that includes pixels for which data saturation has beendetected will lower detection precision. The specified number isappropriately changed in accordance with conditions that have been set(such as a number of pixels constituting a focus detection region, forexample).

If area selection has been performed, next, focused state is determined(S19). Here, the CPU 212 determines whether or not the focus lens 102 ais in a focused state. Specifically, it is determined whether or notfocus deviation amount of the focus detection region that was selectedin the area selection processing is within a predetermined permissiblerange, and a focused state is determined if the focus deviation amountis within the permissible range.

If the result of determination in step S19 is not a focused state, focuslens drive is executed (S21). Here, the CPU 212 performs communicationwith the lens CPU 106 to output a control signal to the lens CPU 106 soas to drive the focus lens 102 a to a focus lens position that wascalculated for the focus detection region that was selected in step S17.Upon receiving this control signal, the lens CPU 106 drives the focuslens 102 a by means of the drive section 104 to a position that has beeninstructed. Once the focus lens 102 a has been driven processing returnsto step S1.

If the result of determination in step S19 is a focused state, exposureand readout for AF & LV are performed (S23). Here, similarly to step S5,the CPU 212 performs imaging (exposure) and readout for autofocus (AF)and live view (LV) display. A pixel signal is read out from the imagesensor 208, focus detection pixel data for AF is stored in the DRAM 226,and display pixel data for LV is stored in the DRAM 226. Also, live view(LV) display is performed using the display pixel data for LV.

Next, correlation calculation and reliability determination areperformed (S25). Here, similarly to step S7, the CPU 212 causesexecution of focus detection calculation by the focus detection circuit218, using pixel data that was read out in step S23. The focus detectioncircuit 218 performs correlation calculation using focus detection pixeldata that constitute a pair, that is stored in the DRAM 226. Aftercorrelation calculation, the focus detection circuit 218 performsreliability determination for focus detection.

If reliability determination has been performed, detection of whether ornot R/L pixel data is saturated is performed (S27), and if the result ofthis determination is that the pixel data is saturated exposurecorrection amount Tv_RL+ is set to 1 (S29). If setting of exposurecorrection amount has been set, or if the result of determination instep S27 is that R/L pixel data is not saturated, next, determination ofwhether or not T/B pixel data is saturated is performed (S31), and ifthe result of this determination is that the T/B pixel data issaturated, exposure correction amount Tv_TB+ is set to 1 (S33). Theprocessing in these steps S27-S33 is similar to the processing in stepsS9-S15 described previously, and so detailed description is omitted.

If exposure correction amount has been set in step S33, or if the resultof determination in step S31 was that T/B pixels were not saturated,then next, focus deviation amount is detected (S35), and area selectionis performed (S37). In steps S35 and S37, similarly to step S17, thefocus detection circuit 218 calculates focus deviation amount (defocusamount) of the focus lens 102 a using a two-image interval value of afocus detection region that was determined to have high reliability atthe time of the reliability determination of step S25, and selects afocus detection region (and corresponding defocus amount) based on thisfocus deviation amount. An example of focus detection region (area) issimilar to that in step S17, and so detailed description is omitted.

Once area selection has been performed, next, history information issaved (S39). Here, the focus detection circuit 218 saves informationrelating to focus detection as history information in the DRAM 226, forexample. Information relating to focus detection includes, for example,information on the focus deviation amount that was calculated in stepS17, pixel data acquisition time, and information on the focus detectionregion that was selected. It should be noted that saving of historyinformation may also be the CPU 212 saving information relating to focusdetection in the DRAM 226.

Once history information has been saved, it is next determined whetheror not the 2nd release switch is on (S41). Here, the CPU 212 determineswhether or not the 2nd release switch within the operation section 206has been turned on. The user presses the release button down fully inthe case of shooting a still picture image for storage. If the userperforms a full-press operation of the release button, the 2nd releaseswitch enters an on state in response to this operation.

If the result of determination in step S41 is that the 2nd releaseswitch is not on, it is next determined whether or not there is afocused state (S43). Here, processing similar to that in step S19 isperformed, and if the result of determination is a focused state,processing returns to S23.

On the other hand, if the result of determination in step S43 is not afocused state, the focus lens is driven (S45). Here, similarly to stepS21, the CPU 212 moves the focus lens 102 a to an appropriate focus lensposition based on the focus deviation amount. If focus lens drive hasbeen performed, processing returns to step S23.

Returning to step S41 if the result of determination in this step isthat the 2nd release switch is on, moving body estimation computation isperformed (S47). Here, the CPU 212 causes execution of moving bodyestimation computation by the focus detection circuit 218. Moving bodyestimation computation is estimating a position where the focus lens 102a will be in focus for the current still picture exposure time, based onhistory of results of previous focus deviation amount calculation (focuslens position) and detection times that were stored in step S39.

If moving body estimation computation has been performed, next a shutteroperation is commenced (S49). Here, the CPU 212 causes commencement ofoperation of the mechanical shutter 202 in order to perform imaging(actual exposure) for still picture acquisition for storage. Thisoperation of the mechanical shutter 202 includes opening and closingoperations of the mechanical shutter 202 before and after actualexposure, and a fully open operation of the mechanical shutter 202 afteractual exposure for the purpose of imaging for live view and AF (in theflow of FIG. 9C, only the time of commencement of the shutter operationis described). If the shutter operation has commenced, the CPU 212 firstswitches control signals of the drive section 204 so as to put themechanical shutter 202 in a fully closed state. Then, in step S53,during actual exposure the mechanical shutter 202 is temporarily closedfully after having been fully open. After that, the CPU 212 controls thedrive section 204 so as to put the mechanical shutter 202 in a fullyopen state.

If the shutter operation has commenced, aperture drive of the apertureand focus lens drive (LD) are simultaneously commenced (S51). Here, theCPU 212 instructs the lens CPU 106 so as to drive the focus lens 102 aand the aperture 102 b at the same time, and both operations arecommenced. Drive position for the focus lens 102 a is a position thatwas estimated in the moving body estimation computation of step S47.Opening amount of the aperture 102 b is an opening amount correspondingto the aperture value that has been calculated based on subjectbrightness that was measured as a result of previous photometrycomputation.

If aperture and focus lens drive have been performed, next, actualexposure is performed (S53). Here, the CPU 212 executes actual exposure,and, as was described previously, controls the drive section 204 so asto cause exposure for a predetermined exposure time using the mechanicalshutter 202. Actual exposure is imaging to acquire image data for stillpicture storage, and with an exposure and readout operation pixelsvalues resulting from having added outputs of four photodiodes for everypixel corresponding to a micro lens (4PD addition) are generated andoutput. After actual exposure is complete, the CPU 212 causes executionof an operation to read out pixel signals from the image sensor 208,after pixel signal readout causes processing for generation of stillpicture data for storage to be performed in the image processing circuit214. The image processing circuit 214 generates still picture data forstorage by performing necessary processing to generate image data forstorage. After completion of image processing, the CPU 212 compressesthe still picture data for storage using the image compression andexpansion section 216. After completion of compression, the CPU 212stores the still picture data for storage that has been compressed inthe storage medium 230 as an image file.

Next, aperture and LD simultaneous drive (opening) is executed (S55).Here, the CPU 212 instructs the lens CPU 106 so as to open the aperture102 b, and so as to drive the focus lens 102 a to a target position. Instep S51, the focus lens 102 a was driven to a position that wasestimated by the moving body estimation computation of step S47.However, there may be times when target position is not reached due tolimitations of processing time for actual exposure, and a remainingdrive amount arises. In step S55 there is an instruction so as to drivethe focus lens by this remaining drive amount.

It is next determined whether or not the 1st release switch is on (S57).Here, the CPU 212 determines whether or not a 1st release switch of arelease button within the operation section 206 is in an on state. Ifthe result of this determination is that the 1st release switch is on,processing returns to step S23 and the previously described processingis executed.

On the other hand, if the result of determination in step S57 is thatthe 1st release switch is not on, it is determined whether or not thecamera power supply is off (S59). Here, the CPU 212 determines whetheror not to turn the power supply of the camera body 200 off. For examplein a case where power supply off has been instructed as a result ofoperation of the operation section 206 by the user, or in a case wherethe user has not operated the operation section 206 for a predeterminedtime, it is determined that the power supply will be turned off. If theresult of this determination is not to turn the power supply of thecamera body 200 off, processing returns to step S1. On the other hand ifit has been determined in step S59 to turn the power supply of thecamera body 200 off, the processing of this flow is terminated.

As has been described above, with the one embodiment of the presentinvention level of a barrier Ptb positioned between LT and LB, andbetween RT and RB of 4 photodiodes (4PD) is set lower than a barrier Prlpositioned between LT and RT, and between LB and RB, giving priority toRL. As a result of this, saturation of electrical charge does not occurin R pixels (RT+RB) or L pixels (LT+LB), even if saturation ofelectrical charge relating to the barrier Ptb occurs in any one of lightreceiving sections LT, LB, RT, or TB. Accordingly, compared to a casewhere barrier Prl and barrier Ptb are the same, it is possible to makecharge storage determination level (saturation determination level) of asignal voltage for R/L pixels higher, and it is possible to expanddynamic range. It should be noted that TB priority may be set instead ofRL priority. The level of the barrier Ptb positioned between LT and LB,and between RT and RB of the 4PDs may also be set higher than the levelof the barrier Prl positioned between LT and RT, and between LB and RB,thus setting TB priority. Also, there maybe a configuration in whicheither an RL priority barrier or a TB priority barrier is set in every4D pixel, and in a case where R/L pixel signals are used an RL priority4PD pixel is selected and used, while if T/B pixel signals are used a TBpriority 4PD pixel is selected and used.

Next a modified example of the one embodiment will be described usingFIG. 10, FIG. 11A and FIG. 11B. FIG. 10 is block diagram including theimage sensor and the focus detection circuit. Description will be givenof a setting method of saturation determination level for digital pixeldata (hereafter called pixel data) after A/D conversion of the pixelsignal, fora case where it is possible to amplify and output a pixelsignal in the image sensor 208. With this example, the control section(focus detection circuit 218) sets amplification factor (also calledgain) of the image sensor 208, and sets saturation determination levelfor pixel data after A/D conversion in accordance with the amplificationfactor that has been set. It should be noted that the control section isnot limited to being within the focus detection circuit 218, and somefunctions may also be realized by other circuits such as the CPU 212.

In FIG. 10, the image sensor 208 is the same as the image sensor 208shown in FIG. 2, with a pixel signal that has been read out from thepixel section 22 being subjected to amplification processing in theanalog processing section 23, and, after being subjected to A/Dconversion in the ADC processing section 24, output as pixel data. Itshould be noted that the memory section 25, horizontal scanning section26, and output section 27 of FIG. 2 have been omitted from FIG. 10.

Also, there are a phase difference pixel column generating section 218a, phase difference pixel data processing section 218 b, and acorrelation calculation section 218 c within the control section (focusdetection circuit 218). The phase difference pixel line generatingsection 218 a has a phase difference pixel line generating circuit, andis a block for outputting added signals output from the image sensor 208as pixel lines of AF area (focus detection region) units. If there are100 AF areas, for example, output of 100 pixel lines is performed.

The phase difference pixel data processing section 218 b has a phasedifference pixel data processing circuit, and performs preprocessing(various correction such as luminance correction, which will bedescribed later) in order to perform correlation calculation. Asaturation detection section 218 ba is provided within the phasedifference pixel data processing section 218 b. This saturationdetection section 218 ba has a saturation detection circuit, and forevery AF area, determines saturation NG in a case where pixel data hasexceeded a saturation determination level, and outputs that result. Thesaturation determination level will be described later using FIG. 11Aand FIG. 11B. The correlation calculation section 218 c has acorrelation calculation circuit, and performs correlation calculationusing output of the phase difference pixel data processing section 218b.

If gain (pixel signal amplification factor) of the image sensor 208 ischanged, saturation level of the pixel signal also changes, and so it isnecessary to change saturation determination level of pixel data, whichis digital output, in accordance with that change. With the imagingdevice 1, an output value of the ADC processing section 24 correspondingto a saturation determination level of a pixel signal when gain of theimage sensor 208 is 1 is stored as saturation determination level forpixel data. Then, the focus detection circuit 218 increases thesaturation determination level by a gain factor if gain is increased. Ina case where value of this saturation determination level exceeds thedynamic range of digital data within the focus detection circuit 218(data variance range), the saturation determination level is topped outat a maximum value of that dynamic range.

Saturation determination level for gain of the image sensor 208 is shownin FIG. 11B, and a graph of change in RL saturation determination levelwith respect to analog gain of the image sensor 208 is shown in FIG.11A. As shown in FIG. 11B, the saturation determination level for R/Lpixel data (refer to S27 in FIG. 9B) and the saturation determinationlevel for T/B pixel data (refer to S31 in FIG. 9B) are different (referto FIG. 4A), and R/L saturation determination level and T/B saturationlevel are stored in accordance with gain. The focus detection circuit218 (saturation detection section 218 ba) changes saturationdetermination level in steps S27 and S31 of FIG. 9B as gain increases,in accordance with FIG. 11B. The saturation detection section 218 basets saturation determination level in accordance with conditions ofpixel data for which whether or not there is saturation will bedetermined, namely whether it is R/L pixel data or T/B pixel data, andgain. It is determined that there is saturation in the event that pixeldata exceeds the saturation determination level. It should be noted thatin FIG. 11A and FIG. 11B, digital data within the focus detectioncircuit 218 is handled as 12 bit data (4096 LSB), and an upper limit fordynamic range is made 3800 LSB as one example.

Next, saturation detection of pixel data that has been subjected toluminance correction, in a case where luminance correction is executedin order to correct light amount distribution characteristics caused byaberration of an optical system, will be described as a second modifiedexample, using FIG. 12. With this example, R/L pixel data that is outputfrom the image sensor 208 is read out, and luminance correctionprocessing is performed on this R/L pixel data to generate data aftercorrection. Whether or not data before correction is saturated isdetermined by comparing pixel data before performing this correctionwith the saturation determination level.

Light amount distribution characteristic is corrected using luminancecorrection values (refer to FIG. 12) that can be derived using lightreceiving sensitivity characteristic of R/L pixels (RT+RB, LT+LB) of theimage sensor 208 and optical characteristics of the photographing lens.Pixel data after correction PIaft can be derived from equation (1)below, using pixel data before correction PIpre and correction value C.The phase difference pixel data processing section 218 b executescalculation of equation (1) below. It should be noted that i is pixelnumber, and correction value C is normally 1.0 to 5.0.

PIaft(i)=PIpre(i)×C(i)  (1)

FIG. 12 shows correction value (luminance correction value) C for R/Lpixels when a particular photographing lens has been attached to thecamera body. In FIG. 12, the horizontal axis represents pixel positionalong the row direction on the image sensor 208, and the vertical axisrepresents correction value C. Correction values for L opening pixels (Lpixels (LT+LB)) and R opening pixels (R pixels (RT+RB)) are left rightsymmetrical with respect to the center of the image sensor 208, due tocharacteristics of the optical system. Correction values C are obtainedfrom previous measurement or from calculation, and stored in memory. Foreach pixel value it is possible to perform luminance correction bycorrecting using equation (1).

The focus detection circuit 218 (saturation detection section 218 ba)determines saturation when pixel data PIpre before luminance correctionexceeds the RL saturation determination level (refer to FIG. 11A andFIG. 11B) (refer to S9 in FIG. 9A, and S27 in FIG. 9B). As was describedpreviously, because error detection and detection precision are loweredif correlation calculation is performed with pixel data that issaturated, detection of AF areas (focus detection regions) that containa specified number or more of saturated pixel data is not possible.Alternatively, correlation calculation may be performed by performingprocessing such as to replace saturated pixel data with other pixel data(for example, surrounding pixel data). The above description relates toR/L pixel data, but the situation is similar for T/B pixel data(RT+LT/RB+LB), where before performing luminance correction usingluminance correction values for T/B pixels stored in memory, saturationis determined using TB saturation determination level (refer to FIG.11B). By performing saturation detection for pixel data before luminancecorrection in this way, high precision focus detection using focusdetection calculation of only appropriate pixel data becomes possible.

Next, a third modified example will be described. With this embodiment,a description will be given for a case where R pixel data and L pixeldata is respectively read out. As a different method to the method ofreading out pixel data, there is a method of, after having performedone-line readout of an R pixel (RT+RB) signal, performing one-linereadout of (R+L) pixels (4PD addition) for the same pixel line. Withthis method it is possible to acquire pixel data for L pixels bycalculating a difference {(R+L)−R} between two sets of pixel data, beingpixel data for R pixels only (R), and pixel data (R+L) corresponding toa signal resulting from having added an R pixel signal and an L pixelsignal. In this case, it is judged whether or not saturationdetermination level has been exceeded for the {(R+L)−R} pixel data.

FIG. 13 in a conceptual diagram of electrical charge of L/R pixels. Forexample, as shown in FIG. 13, in a state where electrical charge CL ofan L pixel is saturated and exceeds a barrier, and charge Cs leaks outinto the R pixel, electrical charge Cs that has overflowed from the Lpixel is accumulated in the R pixel, separately to the charge CR thathas been photoelectrically converted by the R pixel. In this case, sinceelectrical charge is not saturated in the R pixel, it is determined bysaturation determination for the R pixel data that there is notsaturation. However, it is determined by saturation determination for Lpixel data obtained by calculating a difference {(R+L)−R} between (R+L)pixel data and R pixel data that the L pixel data is saturated.Accordingly, it is possible for charge saturation of the L pixel toarise and for saturation NG to be judged, and it is possible to judgethat R pixel data is also not appropriate. In this case also, it ispossible to correctly judge that correlation calculation results usingthis type of RL pixel data is inaccurate.

As has been described above, the detection device of one embodiment ofthe present invention comprises an image sensor provided with aplurality of pixels having a plurality of light receiving sectionscorresponding to a micro lens, with the plurality of light receivingsections being provided in correspondence with a specified plurality ofphase difference detection directions, capable of adding output signalsof the plurality of light receiving sections in accordance with thephase difference detection direction and outputting an added signal(refer, for example, to FIG. 3A and FIG. 3B), and a control section(controller) for performing focus detection or depth detection usingphase difference detection, based on the added signal of the imagesensor (refer, for example, to the focus detection circuit 218 and CPU212 in FIG. 1, and S35 in FIG. 9B). Also, the image sensor makes heightof a potential barrier between the light receiving sections different inaccordance with the plurality of phase difference detection directions(refer, for example, to Ptb and Prl in FIG. 4A). The control section(controller) instructs a specified phase difference detection directionto the image sensor to execute a charge storage operation of the imagesensor, sets charge storage determination level that is differentdepending on phase difference detection direction, and controls thecharge storage operation of the image sensor based on an added signaland the charge storage determination level (refer, for example, to S9and S13 in FIG. 9A, and S27 and S31 in FIG. 9B). As a result the dynamicrange of signals of the phase difference detection pixels can be madelarger, and it becomes possible to improve AF precision and makeoperations high speed.

It should be noted that with the one embodiment of the presentinvention, some or all of the focus detection circuit 218, imageprocessing circuit 214, image compression and expansion section 216,exposure control circuit 220 etc. may be integrated with the CPU 212 andthe peripheral circuitry of the CPU. It is also possible for the focusdetection circuit 218, image processing circuit 214, image compressionand expansion section 216, exposure control circuit 220 etc. to have ahardware structure such as gate circuits that have been generated basedon a programming language that is described using Verilog, and also touse a hardware structure that utilizes software such as a DSP (digitalsignal processor), and also to be respective circuit sections of aprocessor made up of integrated circuits such as an FPGA (FieldProgrammable Gate Array). These approaches may also be suitablycombined. Alternatively, a processor that is constructed with one ormore CPUs may execute functions of each section, by reading out andexecuting computer programs that have been stored in a storage medium.

Also, with the one embodiment of the present invention, a device fortaking pictures has been described using a digital camera, but as acamera it is also possible to use a digital single lens reflex camera, amirrorless camera, or a compact digital camera, or a camera for movieuse such as a video camera, and further to have a camera that isincorporated into a mobile phone, a smartphone a mobile informationterminal, personal computer (PC), tablet type computer, game consoleetc., or a camera for a scientific instrument such as a medical camera(for example, a medical endoscope), or a microscope, an industrialendoscope, a camera for mounting on a vehicle, a surveillance cameraetc. In any event, it is possible to adopt the present invention as longas a device adopts phase difference AF on an image plane.

Also, among the technology that has been described in thisspecification, with respect to control that has been described mainlyusing flowcharts, there are many instances where setting is possibleusing programs, and such programs may be held in a storage medium orstorage section. The manner of storing the programs in the storagemedium or storage section may be to store at the time of manufacture, orby using a distributed storage medium, or they be downloaded via theInternet.

Also, with the one embodiment of the present invention, operation ofthis embodiment was described using flowcharts, but procedures and ordermay be changed, some steps may be omitted, steps may be added, andfurther the specific processing content within each step may be altered.It is also possible to suitably combine structural elements fromdifferent embodiments.

Also, regarding the operation flow in the patent claims, thespecification and the drawings, for the sake of convenience descriptionhas been given using words representing sequence, such as “first” and“next”, but at places where it is not particularly described, this doesnot mean that implementation must be in this order.

As understood by those having ordinary skill in the art, as used in thisapplication, ‘section,’ ‘unit,’ ‘unit,’ ‘component,’ ‘element,’‘module,’ ‘device,’ ‘member,’ ‘mechanism,’ ‘apparatus,’ ‘machine,’ or‘system’ may be implemented as circuitry, such as integrated circuits,application specific circuits (“ASICs”), field programmable logic arrays(“FPLAs”), etc., and/or software implemented on a processor, such as amicroprocessor.

The present invention is not limited to these embodiments, andstructural elements may be modified in actual implementation within thescope of the gist of the embodiments. It is also possible form variousinventions by suitably combining the plurality structural elementsdisclosed in the above described embodiments. For example, it ispossible to omit some of the structural elements shown in theembodiments. It is also possible to suitably combine structural elementsfrom different embodiments.

What is claimed is:
 1. A detection device, comprising: an image sensorprovided with a plurality of pixels having a plurality of lightreceiving sections corresponding to a micro lens, with the plurality oflight receiving sections being provided in correspondence with aspecified plurality of phase difference detection directions, capable ofadding output signals of the plurality of light receiving sections inaccordance with the phase difference detection direction and outputtingan added signal, and a controller for performing focus detection ordepth detection using phase difference detection, based on the addedsignal of the image sensor, whereby the image sensor makes height of apotential barrier between the light receiving sections different inaccordance with the plurality of phase difference detection directions,and the controller executes a charge storage operation of the imagesensor by instructing a specified phase difference detection directionto the image sensor, sets a charge storage determination level thatdiffers in accordance with the phase difference detection direction, andcontrols the charge storage operation of the image sensor based on theadded signal and the charge storage determination level.
 2. Thedetection device of claim 1, wherein: the controller sets the chargestorage determination level in accordance with height of the potentialbarrier corresponding to the phase difference detection direction. 3.The detection device of claim 1, wherein: the controller compares theadded signal with the charge storage determination level, and controlscharge storage operation of the image sensor based on a result ofcomparison.
 4. The detection device of claim 1, wherein: the imagesensor is capable of amplifying and outputting the added signal, and thecontroller sets amplification factor of the image sensor, and sets thecharge storage determination level in accordance with the amplificationfactor that has been set.
 5. The detection device of claim 1, wherein:the controller sets a saturation determination level in accordance withheight of the potential barrier corresponding to the phase differencedetection direction, and determines saturation by comparing the addedsignal output by the image sensor with the saturation determinationlevel.
 6. The detection device of claim 5 wherein: the controllercontrols charge storage operation of the image sensor based on thedetermination result.
 7. An imaging device having the detection deviceof claim 1, the imaging device being capable of continuous shooting ofstill pictures based on an added signal resulting from having added alloutput signals of the plurality of light receiving sections on the imagesensor, whereby: in a case where added signals of a plurality of lightreceiving sections corresponding to phase difference detection directionare acquired by causing imaging by the image sensor, between continuousshooting of still pictures, the controller reads out only added signalscorresponding to a phase difference detection direction for whichpotential barrier has been set higher, as added signals of the pluralityof light receiving sections corresponding to the phase differencedetection direction.
 8. A detection method, for a detection devicehaving an image sensor provided with a plurality of pixels having aplurality of light receiving sections corresponding to a micro lens,with the plurality of light receiving sections being provided incorrespondence with a specified plurality of phase difference detectiondirections, capable of adding output signals of the plurality of lightreceiving sections in accordance with the phase difference detectiondirection and outputting the added signal, the detection methodcomprising: making height of a potential barrier between the lightreceiving sections different in accordance with the plurality of phasedifference detection directions, executing a charge storage operation ofthe image sensor by instructing a specified phase difference detectiondirection to the image sensor, setting a charge storage determinationlevel that differs in accordance with the phase difference detectiondirection, and controlling the charge storage operation of the imagesensor based on the added signal and the charge storage determinationlevel, and performing focus detection or depth detection using phasedifference detection, based on the added signal of the image sensor. 9.The detection method of claim 8, further comprising: setting the chargestorage determination level in accordance with height of the potentialbarrier corresponding to the phase difference detection direction. 10.The detection method of claim 8, further comprising: comparing the addedsignal with the charge storage determination level, and controllingcharge storage operation of the image sensor based on a result ofcomparison.
 11. The detection method of claim 8, wherein: it is possibleto amplify and output the added signal from the image sensor, andfurther comprising setting amplification factor of the image sensor, andsetting the charge storage determination level in accordance with theamplification factor that has been set.
 12. The detection method ofclaim 8, further comprising: setting a saturation determination level inaccordance with height of the potential barrier corresponding to thephase difference detection direction, and determining saturation bycomparing the added signal output by the image sensor with thesaturation determination level.
 13. The detection method of claim 12,further comprising: controlling charge storage operation of the imagesensor based on the determination result.
 14. The detection method ofclaim 8, being provided in an imaging device, the imaging device beingcapable of continuous shooting of still pictures based on an addedsignal resulting from having added all output signals of the pluralityof light receiving sections on the image sensor, wherein the detectionmethod: in a case where added signals of a plurality of light receivingsections corresponding to phase difference detection direction areacquired by causing imaging by the image sensor, between continuousshooting of still pictures, reads out only added signals correspondingto a phase difference detection direction for which potential barrierhas been set higher, as added signals of the plurality of lightreceiving sections corresponding to the phase difference detectiondirection.
 15. A non-transitory computer-readable medium storing aprocessor executable code, which when executed by at least oneprocessor, performs a detecting method, the processor being provided ina detection device, the detection device having an image sensor providedwith a plurality of pixels having a plurality of light receivingsections corresponding to a micro lens, with the plurality of lightreceiving sections being provided in correspondence with a specifiedplurality of phase difference detection directions, capable of addingoutput signals of the plurality of light receiving sections inaccordance with the phase difference detection direction and outputtingan added signal the detecting method comprising: making height of apotential barrier between the light receiving sections different inaccordance with the plurality of phase difference detection directions,executing a charge storage operation of the image sensor by instructinga specified phase difference detection direction to the image sensor,setting a charge storage determination level that differs in accordancewith the phase difference detection direction, and controlling a chargestorage operation of the image sensor based on the added signal and thecharge storage determination level, and performing focus detection ordepth detection using phase difference detection, based on the addedsignal of the image sensor.
 16. The non-transitory computer-readablemedium of claim 15, storing further processor executable code, whichwhen executed by the at least one processor, causes the at least oneprocessor to perform a method further comprising: setting the chargestorage determination level in accordance with height of the potentialbarrier corresponding to the phase difference detection direction. 17.The non-transitory computer-readable medium of claim 15, storing furtherprocessor executable code, which when executed by the at least oneprocessor, causes the at least one processor to perform a method furthercomprising: comparing the added signal with the charge storagedetermination level, and controlling charge storage operation of theimage sensor based on a result of comparison.
 18. The non-transitorycomputer-readable medium of claim 15, storing further processorexecutable code, which when executed by the at least one processor,causes the at least one processor to perform a method, wherein: it ispossible to amplify and output the added signal from the image sensor,and further comprising setting amplification factor of the image sensor,and setting the charge storage determination level in accordance withthe amplification factor that has been set.
 19. The non-transitorycomputer-readable medium of claim 15, storing further processorexecutable code, which when executed by the at least one processor,causes the at least one processor to perform a method furthercomprising: setting a saturation determination level in accordance withheight of a potential barrier corresponding to the phase differencedetection direction, and determining saturation by comparing the addedsignal output by the image sensor with the saturation determinationlevel.
 20. The non-transitory computer-readable medium of claim 19,storing further processor executable code, which when executed by the atleast one processor, cause the at least one processor to perform amethod further comprising: controlling charge storage operation of theimage sensor based on the determination result.